statistically significant effect
From Canada to Japan: How 10,000 km Affect User Perception in Robot Teleoperation
Capy, Siméon, Kwok, Thomas M., Joseph, Kevin, Kawasumi, Yuichiro, Nagashima, Koichi, Sasaki, Tomoya, Hu, Yue, Yoshida, Eiichi
Robot teleoperation (RTo) has emerged as a viable alternative to local control, particularly when human intervention is still necessary. This research aims to study the distance effect on user perception in RTo, exploring the potential of teleoperated robots for older adult care. We propose an evaluation of non-expert users' perception of long-distance RTo, examining how their perception changes before and after interaction, as well as comparing it to that of locally operated robots. We have designed a specific protocol consisting of multiple questionnaires, along with a dedicated software architecture using the Robotics Operating System (ROS) and Unity. The results revealed no statistically significant differences between the local and remote robot conditions, suggesting that robots may be a viable alternative to traditional local control.
- North America > United States (0.16)
- North America > Canada > Ontario > Waterloo Region > Waterloo (0.04)
- Europe > Switzerland (0.04)
- (7 more...)
- Research Report > New Finding (1.00)
- Research Report > Experimental Study > Negative Result (0.48)
Embers of Autoregression: Understanding Large Language Models Through the Problem They are Trained to Solve
McCoy, R. Thomas, Yao, Shunyu, Friedman, Dan, Hardy, Matthew, Griffiths, Thomas L.
The widespread adoption of large language models (LLMs) makes it important to recognize their strengths and limitations. We argue that in order to develop a holistic understanding of these systems we need to consider the problem that they were trained to solve: next-word prediction over Internet text. By recognizing the pressures that this task exerts we can make predictions about the strategies that LLMs will adopt, allowing us to reason about when they will succeed or fail. This approach - which we call the teleological approach - leads us to identify three factors that we hypothesize will influence LLM accuracy: the probability of the task to be performed, the probability of the target output, and the probability of the provided input. We predict that LLMs will achieve higher accuracy when these probabilities are high than when they are low - even in deterministic settings where probability should not matter. To test our predictions, we evaluate two LLMs (GPT-3.5 and GPT-4) on eleven tasks, and we find robust evidence that LLMs are influenced by probability in the ways that we have hypothesized. In many cases, the experiments reveal surprising failure modes. For instance, GPT-4's accuracy at decoding a simple cipher is 51% when the output is a high-probability word sequence but only 13% when it is low-probability. These results show that AI practitioners should be careful about using LLMs in low-probability situations. More broadly, we conclude that we should not evaluate LLMs as if they are humans but should instead treat them as a distinct type of system - one that has been shaped by its own particular set of pressures.
- Europe > United Kingdom > England > Cambridgeshire > Cambridge (0.14)
- North America > United States > Minnesota > Hennepin County > Minneapolis (0.13)
- Europe > United Kingdom > England > Oxfordshire > Oxford (0.13)
- (18 more...)
- Research Report > New Finding (1.00)
- Research Report > Experimental Study (1.00)
- Information Technology (0.67)
- Government > Regional Government > North America Government > United States Government (0.67)
- Education (0.67)
- Health & Medicine > Therapeutic Area > Neurology (0.45)